Applying Decision Theory to Reactive Planning

We are engaged in a research project aimed at the synthesis of reactive plans [BDg0, GK91, Kabg0, Mitg0, DKKN93a]. Reactive plans are different from classical plans in that they are specified as a collection of rules or "reactions" of the form S =~ a, where S is a description (perhaps partial) of a situation or state and is an action executable by the agent (such rules have also been called situated control rules by Drummond [DruB9]). The meaning of such a rule is that if the agent finds itself in a state characterized by S, then it should execute action a. Hence, a key difference between classical plans and reactive plans is that reactive plans contain no notion of a predetermined action execution sequence. Rather, the sequence of actions executed will depend on what happens in the agent’s environment. This makes reactive plans more successful in real environments where the agent’s actions are not the only source of change. Another key difference is that a classical plan, i.e., a finite sequence of actions to be executed by the agent, can only achieve a limited set of goals. For the most part only simple goals of achievement have been studied in this context. As we will discuss in more detail below, a reactive plan can be used to achieve certain types of temporally extended goals that are beyond the capability of simple classical p ans. Unless we take the approach of universal p ans [Sch87] where we store a reaction for every possible situation, we need to be able to compute reactions dynamically. The ability to do reactive planning in this way has a number of benefits. First, it allows the agent to adapt to its environment and to changes in its environment by replacing its reactions with newly computed superior reactions. Second, the reactive planner can easily provide an anytime capability by always retaining its old reactions until it can replace them with better ones. And third, a reactive planner allows us to specify new goals to the agent and have it alter its reactions so as to achieve these new goals. There are three main components of the planning architecture we are investigating: (1) a theory of action, (2) a language for expressing goals, and (3) our approach